Improved physiological noise regression in fNIRS: a multimodal extension of the General Linear Model using temporally embedded Canonical Correlation Analysis

For the robust estimation of evoked brain activity from functional Near-Infrared Spectroscopy (fNIRS) signals, it is crucial to reduce nuisance signals from systemic physiology and motion. The current best practice incorporates short-separation (SS) fNIRS measurements as regressors in a General Linear Model (GLM). However, several challenging signal characteristics such as non-instantaneous and non-constant coupling are not yet addressed by this approach and additional auxiliary signals are not optimally exploited. We have recently introduced a new methodological framework for the unsupervised multivariate analysis of fNIRS signals using Blind Source Separation (BSS) methods. Building onto the framework, in this manuscript we show how to incorporate the advantages of regularized temporally embedded Canonical Correlation Analysis (tCCA) into the supervised GLM. This approach allows flexible integration of any number of auxiliary modalities and signals. We provide guidance for the selection of optimal parameters and auxiliary signals for the proposed GLM extension. Its performance in the recovery of evoked HRFs is then evaluated using both simulated ground truth data and real experimental data and compared with the GLM with short-separation regression. Our results show that the GLM with tCCA significantly improves upon the current best practice, yielding significantly better results across all applied metrics: Correlation (HbO max. +45%), Root Mean Squared Error (HbO max. -55%), F-Score (HbO up to 3.25-fold) and p-value as well as power spectral density of the noise floor. The proposed method can be incorporated into the GLM in an easily applicable way that flexibly combines any available auxiliary signals into optimal nuisance regressors. This work has potential significance both for conventional neuroscientific fNIRS experiments as well as for emerging applications of fNIRS in everyday environments, medicine and BCI, where high Contrast to Noise Ratio is of importance for single trial analysis.Published versio

Wearable, high-density fNIRS and diffuse optical tomography technologies: a perspective

Recent progress in optoelectronics has made wearable and high-density functional near-infrared spectroscopy (fNIRS) and diffuse optical tomography (DOT) technologies possible for the first time. These technologies have the potential to open new fields of real-world neuroscience by enabling functional neuroimaging of the human cortex at a resolution comparable to fMRI in almost any environment and population. In this perspective article, we provide a brief overview of the history and the current status of wearable high-density fNIRS and DOT approaches, discuss the greatest ongoing challenges, and provide our thoughts on the future of this remarkable technology

Using the General Linear Model to Improve Performance in fNIRS Single Trial Analysis and Classification: A Perspective

Within a decade, single trial analysis of functional Near Infrared Spectroscopy (fNIRS) signals has gained significant momentum, and fNIRS joined the set of modalities frequently used for active and passive Brain Computer Interfaces (BCI). A great variety of methods for feature extraction and classification have been explored using state-of-the-art Machine Learning methods. In contrast, signal preprocessing and cleaning pipelines for fNIRS often follow simple recipes and so far rarely incorporate the available state-of-the-art in adjacent fields. In neuroscience, where fMRI and fNIRS are established neuroimaging tools, evoked hemodynamic brain activity is typically estimated across multiple trials using a General Linear Model (GLM). With the help of the GLM, subject, channel, and task specific evoked hemodynamic responses are estimated, and the evoked brain activity is more robustly separated from systemic physiological interference using independent measures of nuisance regressors, such as short-separation fNIRS measurements. When correctly applied in single trial analysis, e.g., in BCI, this approach can significantly enhance contrast to noise ratio of the brain signal, improve feature separability and ultimately lead to better classification accuracy. In this manuscript, we provide a brief introduction into the GLM and show how to incorporate it into a typical BCI preprocessing pipeline and cross-validation. Using a resting state fNIRS data set augmented with synthetic hemodynamic responses that provide ground truth brain activity, we compare the quality of commonly used fNIRS features for BCI that are extracted from (1) conventionally preprocessed signals, and (2) signals preprocessed with the GLM and physiological nuisance regressors. We show that the GLM-based approach can provide better single trial estimates of brain activity as well as a new feature type, i.e., the weight of the individual and channel-specific hemodynamic response function (HRF) regressor. The improved estimates yield features with higher separability, that significantly enhance accuracy in a binary classification task when compared to conventional preprocessing—on average +7.4% across subjects and feature types. We propose to adapt this well-established approach from neuroscience to the domain of single-trial analysis and preprocessing wherever the classification of evoked brain activity is of concern, for instance in BCI

Towards a wireless open source instrument: functional Near-Infrared Spectroscopy in mobile neuroergonomics and BCI applications

Brain-Computer Interfaces (BCIs) and neuroergonomics research have high requirements regarding robustness and mobility. Additionally, fast applicability and customization are desired. Functional Near-Infrared Spectroscopy (fNIRS) is an increasingly established technology with a potential to satisfy these conditions. EEG acquisition technology, currently one of the main modalities used for mobile brain activity assessment, is widely spread and open for access and thus easily customizable. fNIRS technology on the other hand has either to be bought as a predefined commercial solution or developed from scratch using published literature. To help reducing time and effort of future custom designs for research purposes, we present our approach toward an open source multichannel stand-alone fNIRS instrument for mobile NIRS-based neuroimaging, neuroergonomics and BCI/BMI applications. The instrument is low-cost, miniaturized, wireless and modular and openly documented on www.opennirs.org. It provides features such as scalable channel number, configurable regulated light intensities, programmable gain and lock-in amplification. In this paper, the system concept, hardware, software and mechanical implementation of the lightweight stand-alone instrument are presented and the evaluation and verification results of the instrument\u27s hardware and physiological fNIRS functionality are described. Its capability to measure brain activity is demonstrated by qualitative signal assessments and a quantitative mental arithmetic based BCI study with 12 subjects

Simultaneous Acquisition of EEG and NIRS during Cognitive Tasks for an Open Access Dataset

We provide an open access multimodal brain-imaging dataset of simultaneous electroencephalography (EEG) and near-infrared spectroscopy (NIRS) recordings. Twenty-six healthy participants performed three cognitive tasks: 1) n-back (0-, 2- and 3-back), 2) discrimination/selection response task (DSR) and 3) word generation (WG) tasks. The data provided includes: 1) measured data, 2) demographic data, and 3) basic analysis results. For n-back (dataset A) and DSR tasks (dataset B), event-related potential (ERP) analysis was performed, and spatiotemporal characteristics and classification results for “target” vs. “non-target” (dataset A) and symbol “O” vs. symbol “X” (dataset B) are provided. Time-frequency analysis was performed to show the EEG spectral power to differentiate the task-relevant activations. Spatiotemporal characteristics of hemodynamic responses are also shown. For the WG task (dataset C), the EEG spectral power and spatiotemporal characteristics of hemodynamic responses are analyzed, and the potential merit of hybrid EEG-NIRS BCIs was validated with respect to classification accuracy. We expect that the dataset provided will facilitate performance evaluation and comparison of many neuroimaging analysis techniques

Towards neuroscience of the everyday world (NEW) using functional near infrared spectroscopy

Published in final edited form as: Curr Opin Biomed Eng. 2021 June ; 18: doi:10.1016/j.cobme.2021.100272.Functional near-infrared spectroscopy (fNIRS) assesses human brain activity by noninvasively measuring changes of cerebral hemoglobin concentrations caused by modulation of neuronal activity. Recent progress in signal processing and advances in system design, such as miniaturization, wearability, and system sensitivity, have strengthened fNIRS as a viable and cost-effective complement to functional magnetic resonance imaging, expanding the repertoire of experimental studies that can be performed by the neuroscience community. The availability of fNIRS and electroencephalography for routine, increasingly unconstrained, and mobile brain imaging is leading toward a new domain that we term “Neuroscience of the Everyday World” (NEW). In this light, we review recent advances in hardware, study design, and signal processing, and discuss challenges and future directions.U01EB029856 - National Institutes of HealthAccepted manuscrip

Multimodale Instrumente und Methoden für Neurotechnologie außerhalb des Labors

In neuroscience and related fields, progress in instrumentation, computational power, and signal processing methods continuously provide novel and increasingly powerful tools toward the investigation of brain activity in real-time and everyday environments. Research into real-life and application-oriented, non-invasive neurotechnology bears a number of multidisciplinary challenges which need to be addressed. Neurophysiological signals have to be measured subtly and safely while reliability and robustness have to be ensured. To this end, new approaches are explored in this thesis that deal with the simultaneous acquisition and utilization of multiple brain and body signals in mobile scenarios. They aim to reduce acquisition restraints for mobile neuroimaging, and at the same time increase the amount of information that is provided by hybrid acquisition equipment. This enables the exploitation of complementary and shared information in the measured modalities toward the development of methods that enhance robustness in the analysis and classification of brain signals. The first contribution of this work comprises the development of novel architectures and devices for the mobile measurement of brain and body signals. Here, the focus lies on functional Near-Infrared Spectroscopy (fNIRS) and Electroencephalography (EEG) instruments. The primary result is M3BA, an architecture for Mobile, Modular, Multimodal Biosignal Acquisition. While miniaturized beyond previous approaches, M3BA offers hybrid and high-precision measurement of fNIRS, EEG, acceleration and other signals while allowing scalability and easy customization. The second contribution targets the generation of evoked multimodal neuroimaging data under realistic environmental but yet well-controlled movement conditions. Making use of M3BA modules in a lightweight wireless headset, a novel, bespoke n-back-based cognitive workload paradigm was designed and administered in a study with 17 freely moving subjects. Using this unique dataset, the third contribution consists of the development of a multimodal Blind-Source-Separation framework for the analysis of fNIRS signals and its application in BLISSA2RD, for the accelerometer-based rejection of movement induced artifacts. Employing it along with other state-of-the-art methods, we ultimately provide a proof of feasibility toward workload classification under challenging, realistic conditions. In this unique approach, and with strict rejection of artifacts, accuracies greater than 80% based on neurophysiological EEG-fNIRS markers is achieved.In den Neurowissenschaften und ihren angrenzenden Feldern ermöglichen Fortschritte in der Messtechnik, Miniaturisierung, Rechenleistung und Signalverarbeitung leistungsstarke Ansätze zur Untersuchung der Gehirnaktivität in Echtzeit und unter zunehmend alltagsähnlichen Bedingungen. Die Erforschung nicht-invasiver Neurotechnologie für anwendungsorientierte Szenarien außerhalb des Labors birgt jedoch eine Vielzahl multidisziplinärer Herausforderungen. Neuartige Ansätze müssen eine unaufdringliche und schadlose Erfassung neurophysiologischer Signale ermöglichen und gleichzeitig Zuverlässigkeit und Robustheit sicher stellen. Zu diesem Zweck werden in dieser Dissertation neue Ansätze untersucht, die sich mit der simultanen Erfassung und Nutzung von multiplen Gehirn- und Körpersignalen in mobilen Szenarien beschäftigen. Durch die Verbindung von Biomedizintechnik, Neurowissenschaften und Maschinellem Lernen sollen die Möglichkeiten bei der Signalerfassung erweitert und die Menge der erfassten Informationen erhöht werden. Diese ermöglicht die Entwicklung multimodaler Methoden zur Verbesserung von Signalqualität und Robustheit. Der erste Teil dieser Arbeit besteht aus der Entwicklung von Grundlagen und Architekturen für den Entwurf neuer Instrumente zur mobilen, miniaturisierten und hybriden Messung von Gehirn- und peripheren Körpersignalen. Dabei liegen die Schwerpunkte auf der funktionellen Nahinfrarot-Spektroskopie (fNIRS) und Elektroenzephalographie (EEG). Das primäre Resultat ist M3BA, eine Mobile, Modulare, Multimodale Biosignalerfassungs-Architektur. Während M3BA gegenüber früherer Ansätze weiter miniaturisiert ist, bietet es hochpräzise hybride fNIRS-EEG- und Accelerometer-Messungen, Skalierbarkeit und einfache Anpassung. Diese Architektur ermöglicht im zweiten Teil die Entwicklung und experimentelle Umsetzung eines neuartigen räumlichen n-back-Paradigmas für die Erfassung der mentalen Arbeitslast in sich frei bewegenden Teilnehmern. Der resultierende Datensatz, mit einem speziell darauf ausgelegten M3BA-Headgear erfasst, bietet eine Vielfalt physiologischer Signale von 17 Probanden unter kontrollierten Bewegungsbedingungen. Unter Verwendung dieses neuen Datensatzes besteht der dritte Teil aus der Entwicklung einer Methode zur Analyse von fNIRS-Signalen und der Accelerometer-basierten Entfernung von fNIRS Bewegungsartefakten mit dem Namen BLISSA2RD. In Kombination dieser Methode mit anderen state-of-the art Ansätzen und unter strikter Artefaktbereinigung wird abschließend die Klassifizierung mentaler Arbeitslast unter herausfordernden, realitätsnahen Bedingungen untersucht. Ein erster Machbarkeitsnachweis wird mit erreichten Klassifikationsgenauigkeiten von > 80% unter Ausnutzung der Multimodalität der Daten erbracht

Optical brain imaging and its application to neurofeedback

Besides passive recording of brain electric or magnetic activity, also non-ionizing electromagnetic or optical radiation can be used for real-time brain imaging. Here, changes in the radiation’s absorption or scattering allow for continuous in vivo assessment of regional neurometabolic and neurovascular activity. Besides magnetic resonance imaging (MRI), over the last years, also functional near-infrared spectroscopy (fNIRS) was successfully established in real-time metabolic brain imaging. In contrast to MRI, fNIRS is portable and can be applied at bedside or in everyday life environments, e.g., to restore communication and movement. Here we provide a comprehensive overview of the history and state-of-the-art of real-time optical brain imaging with a special emphasis on its clinical use towards neurofeedback and brain-computer interface (BCI) applications. Besides pointing to the most critical challenges in clinical use, also novel approaches that combine real-time optical neuroimaging with other recording modalities (e.g. electro- or magnetoencephalography) are described, and their use in the context of neuroergonomics, neuroenhancement or neuroadaptive systems discussed

Introduction to the shared near infrared spectroscopy format

SIGNIFICANCE: Functional near-infrared spectroscopy (fNIRS) is a popular neuroimaging technique with proliferating hardware platforms, analysis approaches, and software tools. There has not been a standardized file format for storing fNIRS data, which has hindered the sharing of data as well as the adoption and development of software tools. AIM: We endeavored to design a file format to facilitate the analysis and sharing of fNIRS data that is flexible enough to meet the community’s needs and sufficiently defined to be implemented consistently across various hardware and software platforms. APPROACH: The shared NIRS format (SNIRF) specification was developed in consultation with the academic and commercial fNIRS community and the Society for functional Near Infrared Spectroscopy. RESULTS: The SNIRF specification defines a format for fNIRS data acquired using continuous wave, frequency domain, time domain, and diffuse correlation spectroscopy devices. CONCLUSIONS: We present the SNIRF along with validation software and example datasets. Support for reading and writing SNIRF data has been implemented by major hardware and software platforms, and the format has found widespread use in the fNIRS community